Embodiments of the present application relate to the storage of data and analysis of stored data. More particularly embodiments of the present application relate to the storage of data in log-files, and the analysis of data stored in log-files.
In computing, a log-file is a file that lists events that have occurred. For example, a server such as a web server may maintain a log file that lists every request made to the server. A log-file can be useful as it enables performance information to be analysed by an administrator of the server. For example it may enable an administrator of the server to determine whether there are any bugs or anomalies occurring.
A server may have to deal with very large amounts of information. Depending on the context of use, a server may have to deal with thousands, hundreds of thousands, or even millions of events a day. For example a server which hosts an online gaming community may have to deal with many events involving players playing the online game. It would be highly impractical for the data to be manually analysed to detect any anomalies occurring. Therefore the data needs to be analysed in an automated manner. To reduce processing and memory requirements and /or loads on the server, embodiments of the present application may analyse this data in a resource-efficient manner, as explained below.